\chapter{Simulations and findings}
\label{chap:simAndFindings}

\begin{flushright}{\slshape    
   Science, my boy, is made up of mistakes, but they are mistakes
   which it is useful to make, because they lead little by little
   to the truth}. \\ \medskip --- \citeauthor{verne_journey:1957}
   \citetitle{verne_journey:1957}
\end{flushright} 

The reader should now be aware of all the concepts, tools and
methods hidden behind the multi-objective framework detailed in
the previous chapters.
This chapter constitutes the last point before concluding with
some final remarks and open issues.
In fact the results obtained by running our framework with
several experiments are now exposed. The hypotheses onto which
this thesis is built on, explained in previous chapters are now
going to be tested through the analysis of the results of the
experiments.

\section{Overview on the experiments performed}
\label{sec:experiments}
Three main experiments were carried out. 
The logic order adopted for presenting them is the same adopted
during the research. Each experiment is performed with the goal of
bringing some improvements to the results of the previous ones.
The three experiments will be called from this point on
\myExpOne, \myExpTwo and \myExpThree.
Here they are briefly described: 
\begin{description}
  \item [\myExpOne] a multi-objective \ac{GLE} is performed with
  our framework, including the four objectives \ac{TEE}, \ac{EE},
  \ac{EEL} and \ac{EEE}.
  The hydrological parameters adopted by \citeauthor{paik:2011}
  in \cite{paik:2011} are used, \ie rainfall equal to $0.1$ mm/h
  and runoff coefficient equal to $1.0$.
  \item [\myExpTwo] this experiment is equal to the previous one
  but for the input rainfall parameter that is increased from
  $0.1$ mm/h to $10.0$ mm/h. Moreover, the range of possible values 
  which control variables can assume is increased;
  \item [\myExpThree] \ac{IDW} spatial interpolator is added to
  the model of experiment \myExpTwo.
\end{description}

More precisely, the common model settings of the three experiments
are summarized in \myTabNoSpace{tab:experimentsCommonAspects},
their model settings differences are written in
\myTabNoSpace{tab:experimentsDifferences}, while the different
optimization settings for them are listed in
\myTabNoSpace{tab:experimentsEAsettings}.

\begin{table}[h]
\myfloatalign
\footnotesize
\begin{tabularx}{\textwidth}{p{0.55\textwidth}X}
\toprule
\tableheadline{c}{Common feature} & \tableheadline{c}{Description
/ value}\\
\midrule
Optimality principles & \ac{TEE} \ac{EE} \ac{EEL} \ac{EEE}\\
\midrule 
Initial \ac{DEM} shape & square pyramid (top elevation $100$ masl)\\
\midrule
\ac{DEM} cell dimension & 	$1$ km $\times$ $1$ km\\
\midrule
\ac{DEM} number of cells &  $51\times 51$\\
\midrule
Elevations sum conservation tolerance & $0.001$ (\ie $1$\textperthousand)\\
\midrule
Depression filling algorithm & Planchon's \ac{DF}\\
\midrule
Flow directions algorithm & \ac{GD8}\\
\bottomrule
\end{tabularx}
\caption[Common model settings for \myExpOne, \myExpTwo and
\myExpThree experiments] {Common model settings for all the three
analyzed experiments: \myExpOne, \myExpTwo and \myExpThreeNoSpace.}
\label{tab:experimentsCommonAspects}
\end{table}

\begin{table}[h]
\myfloatalign
\footnotesize
\begin{tabularx}{\textwidth}{p{0.2\textwidth}ccc}
\toprule
& \tableheadline{c}{\myExpOne} & \tableheadline{c}{\myExpTwo} &
\tableheadline{c}{\myExpThree} \\
\midrule
\tablefirstcol{p{0.2\textwidth}}{Rainfall (mm/h)} & $0.1$ & 
$10.0$ & $10.0$ \\
\midrule
\tablefirstcol{p{0.2\textwidth}}{$z$ range (m)} &
$(-10;+10)$ & $(-20;+20)$ & $(0; 400)$ \\
\midrule
\tablefirstcol{p{0.2\textwidth}}{$z$ discretization (cm)}
& $100$ & $2$ & $2$ \\
\midrule
\tablefirstcol{p{0.2\textwidth}}{Interpolator} & - & - &
\ac{IDW}\\
\bottomrule
\end{tabularx}
\caption[Differences in model settings for \myExpOne, \myExpTwo
and \myExpThree experiments] {Different model settings for all the
three analyzed experiments: \myExpOne, \myExpTwo and
\myExpThreeNoSpace.}
\label{tab:experimentsDifferences}
\end{table}
	
\begin{table}[h]
\myfloatalign
\footnotesize
\begin{tabularx}{\textwidth}{p{0.3\textwidth}ccc}
\toprule
\tableheadline{c}{Experiment} & \tableheadline{c}{Genetic
Algorithm} & \tableheadline{c}{Seeds} &
\tableheadline{c}{NFE}
\\
\midrule 
\multirow{3}*{\myExpOne} & \ac{eNSGAII} & $2$ & $10$M \\
						 & \ac{GDE3} & $3$ & $10$M \\
						 & Random &  $5$ & $10$M \\
\midrule 
\multirow{3}*{\myExpTwo} & \ac{eNSGAII} & $2$ & $10$M \\
						 & \ac{GDE3} & $4$ & $10$M \\
						 & Random &  $5$ & $10$M \\
\midrule 
\multirow{3}*{\myExpThree} & \ac{eNSGAII} & $11$ & $10$M \\
							 & \ac{GDE3} & $11$ & $10$M \\
						 	& \ac{OMOPSO} &  $10$ & $10$M \\
\bottomrule
\end{tabularx}
\caption[Differences in optimization settings for \myExpOne,
\myExpTwo and \myExpThree experiments] {Different optimization settings for
\myExpOne, \myExpTwo and \myExpThree experiments.
In the column SEEDS, the number of initial random seeds for each
algorithm is listed.
\ac{NFE} stands for \acl{NFE}.}
\label{tab:experimentsEAsettings}
\end{table}

In addition to the information in
\myTabNoSpace{tab:experimentsEAsettings}, the following two
parameters were set, in the same way for each algorithm:
\begin{itemize}
  \item population size of $500$ individuals.
  It represents the number of initial individuals for the
  \ac{GA}s;
  \item $\varepsilon$ values for $\varepsilon$-box widths equal to
  $100.0$ for \ac{TEE}, $100.0$ for \ac{EEL}, $0.1$ for \ac{EE}
  and $0.001$ for \ac{EEE}.
\end{itemize}

\subsection{Computational effort}
One last important comment which is due before starting the real
analysis of the results is about the computational effort which
was required to perform the just described experiments,
both in term of time spent and number of operations executed.
As it is possible to read from \myTabNoSpace{tab:experimentsEAsettings},
a total number of $530$ millions of \ac{NFE} run for the
three experiments we are commenting in this chapter, requiring a
total execution time of over $1400$ hours (about two months!).
In addition, other preliminary and testing experiments were
performed, for an additional execution time of approximately $326$
hours.\footnote{All experiments were run on Lion-XO, a
computational cluster run by the High Performance Computing Group
(PennState University) with a total number of $192$ processors
\emph{AMD Opteron 250 2.4 GHz} and \emph{AMD Opteron 852 2.6 GHz}
and a total RAM of $768$ GB.}

\section{First experiment results: \myExpOne}
\label{sec:expPaik}
This preliminary experiment, as said, was performed using the same
hydrological parameters used in \cite{paik:2011} and considering
the four selected formulations of the optimality principle.
A control variable corresponds to each \ac{DEM} cell and makes it
change its elevations. 
In particular, the range for control variables was set equal to
$[-10 ; +10]$ meters\footnote{The interval was chosen in order to
have a good theoretical feasibility, which, according to
\myFigNoSpace{fig:massConstraintFeasibility}, should be between
$30\%$ and $40\%$, with such a control range.}, and the
discretization step of this interval was set to $1$ m (the same
discretization step used by \citeauthor{paik:2011} in his \ac{GLE}
model.

\subsection{Pareto front}
The Pareto front obtained as a result of the optimization of this
experiment is the one represented in \myFigNoSpace{fig:pfMogle}.
\begin{figure}
\myfloatalign
\includegraphics[width=\columnwidth]{Images/PF_MOGLE.pdf}  
\caption[Pareto front obtained from \myExpOne
experiment]{Pareto front obtained from \myExpOne experiment.
The front was represented using AeroVis\copyright{}.}
\label{fig:pfMogle}
\end{figure}

In the figure, it is possible to see that the Pareto front 
has four dimensions, due to the choice of four criteria to test: 
three are given by $x$, $y$ and $z$ axis, while the fourth one is 
represented through a color bar. 
All the points composing the front are therefore optimal, with
respect to the the four objectives of the minimization problem. 
The fact that they appear spread into the 4 dimensional space
might suggest that, among them, some kind of conflict exists. 
If no conflict existed, probably a single optimum point would be
found, in spite of the Pareto front. 
The topic of conflict will be treated soon, but firstly some
technical features of the front are described.  

\subsubsection{Algorithms performance}
\MyTab{tab:paretoFrontExp1Data} provides some information on the just shown 
front.
In particular, the number of points of the front each algorithm
contributed to is shown, together with the feasibility.
As for feasibility, it represents the number of control sets
respecting the mass constraint over the total number of generated
sets, for each algorithm in the first three rows and for the total
number of function evaluations in the last row.
\begin{table}[h]
\myfloatalign
\begin{tabular}{lSS} 
\toprule {ALGORITHM} & {NUMBER OF FRONT POINTS} & {FEASIBILITY
($\%$)} \\
\midrule
{\ac{eNSGAII}} & 1103 & 58.70\\
\midrule
{\ac{GDE3}} & 102 & 64.44\\
\bottomrule
{\textbf{Total}} & 1205 & 61.57\\ 
\bottomrule
\end{tabular}
\caption[\myExpOneNoSpace: Pareto front
features]{\myExpOneNoSpace: Pareto front features.}
\label{tab:paretoFrontExp1Data}
\end{table}

The largest contribution is therefore coming from \ac{eNSGAII}
algorithm, covering the $91.5\%$ of the front, and the feasibility
of the two algorithms are comparable.
Since the average feasibility is higher than $60\%$, it is
possible to say that the two algorithms are quite good at
recognizing the mass constraint and find feasible control sets.

\subsubsection{Comparison with Random search}
It is possible to assess the effectiveness of the use of \acp{EA}
also by comparing their results to the ones obtained with a random
search algorithm.
As written in \myTabNoSpace{tab:experimentsEAsettings}, this
experiment was executed also running random search algorithm on
five seeds.
The result, which has to be compared with \acp{EA} performance, is
the following:
\begin{itemize}
  \item the capability of the random search algorithm to find
  control sets which respect mass constraint is around $3.38\%$,
  which is very low if compared to the one just mentioned of \acp{EA}, higher than $60\%$.
  This means that \ac{EA} permorm better in understanding the mass constrain;
  \item the feasible points found with random search algorithm can be compared 
  with the ones produced by \acp{EA}, in the objective space.
  As it is represented in \myFigNoSpace{fig:random_vs_mogle}, two fronts are compared:
  the blue front is the one presented at the beginning of this section, while the red one is 
  computed considering the feasible points of random search. 
  It clearly appears that the red front is dominated by the blue one: that means that 
  \acp{EA} perform better than random search as for the minimization problem.
  Moreover, again due to the higher feasibility of \ac{EA}, the number of points in the 
  blue front is larger than the one of the red ones, which are only $80$.
\end{itemize}
The use of \ac{EA} seems then justified.

\begin{figure}
\myfloatalign
\includegraphics[width=0.8\columnwidth]{Images/random_vs_mogle.pdf}
\caption[\myExpOne: Pareto front obtained  with \acp{EA} 
compared with the one obtained with Random search]{\myExpOne: Pareto front obtained  with \acp{EA} 
compared with the one obtained with Random search.
The front was represented using AeroVis\copyright{}.}
\label{fig:random_vs_mogle}
\end{figure}

\subsection{Analysis of the conflict} 
It appears from the front, even if it is not very clear, that a
possible conflicting situation might intercur between \ac{TEE} and
\ac{EE} objectives. It is highlighted in
\myFigNoSpace{fig:MOGLE_2Dproj_TEEvsEE}.
This conflict can be expected and explained, if
\citeauthor{rodriguez:1992} considerations are balanced:
\blockcquote[see][p. 1098]{rodriguez:1992}{
If the rate of energy expenditure per unit area of channel is
maintained the same [\ldots], then the minimum total rate of energy expenditure,
corresponds to the network with the largest $\Omega$.[\ldots] If
one maintains the total rate of energy expenditure, as a constant
in all cases [\ldots] implies a rate of energy expenditure per
unit area of channel largest for the network with the minimum
$\Omega$ and smallest for the network with the minimum $\Omega$}.
In fact, since in our model \ac{TEE} is the total rate of energy
expenditure and \ac{EE} is the rate of energy expenditure per unit
area of channel, it seems that the theory hypothesizes a conflict
between them.\footnote{$\Omega$ is the order of basin network
which for a fixed number of sources is also a measure of the
degree of branching.}

\begin{figure}
\myfloatalign
\includegraphics[width=0.8\columnwidth]{Images/MOGLE_2Dproj_TEEvsEE.pdf}
\caption[Pareto front view on the plane (TEE; EE) for
\myExpOne experiment]{Pareto front view on the plane (TEE; EE)
for \myExpOne experiment. The front was represented using AeroVis
\copyright{}.}
\label{fig:MOGLE_2Dproj_TEEvsEE}
\end{figure}

Furthermore, also the relation between \ac{TEE} and \ac{EEL} is
not clear: it is not possible to clearly establish whether they
are conflicting or not. Their behaviour is highlighted in
\myFigNoSpace{fig:MOGLE_2Dproj_TEEvsEEL}.

\begin{figure}
\myfloatalign
\includegraphics[width=0.8\columnwidth]{Images/MOGLE_2Dproj_TEEvsEEL.pdf}
\caption[Pareto front view on the plane (TEE; EEL) for
\myExpOne experiment]{Pareto front view on the plane (TEE;
EEL) for \myExpOne experiment. The front was represented using AeroVis
\copyright{}.}
\label{fig:MOGLE_2Dproj_TEEvsEEL}
\end{figure}

One evident thing, on the other hand, is that \ac{EE} and \ac{EEE} are no conflicting.
For this fact, it is possible, for the next analysis, \ie clustering, to 
consider just three objectives, in order to reduce the dimensionality of the objectives 
space, keeping into account that minimizing \ac{EE} means also minimizing \ac{EEE}.

\subsection{Clustering and naturality indexes analysis}
In this experiment, five clusters were obtained with $k$-means
clustering and are represented with different colors in
\myFigNoSpace{fig:clusters_MOGLE}.

\begin{figure}
\myfloatalign
\includegraphics[width=0.8\columnwidth]{Images/clusters_MOGLE.pdf}  
\caption[Pareto front clusters for \myExpOne experiment]{Pareto
front clusters for \myExpOne experiment. The upper colorbar
shows the colors of the different $5$ clusters.
Cluster \emph{Min\ac{TEE}} is the blue one, \emph{Compromise} is the 
yellow one and \emph{Min\ac{EE}} the green one.
They are selected for naturality indexes analysis.
The front was represented using AeroVis \copyright{}.}
\label{fig:clusters_MOGLE}
\end{figure}

Despite the fact that conflict appears to be a bit confused,
in order to simplify the analysis, only three clusters were selected
for carrying out the assessment through naturality indexes.
They were chosen in order to have two extreme
values and one compromise among \ac{TEE} and \ac{EE} objectives.
They are the following:
\begin{description}
  \item[\NoCaseChange{MinTEE}:] this cluster minimizes \ac{TEE}
  objective, counting against \ac{EE} objective;
  \item[\NoCaseChange{Compromise}:] this cluster represents the
  compromise between \ac{TEE} and \ac{EE} objectives;
  \item[\NoCaseChange{MinEE}:] this cluster minimizes \ac{EE}
  objective, counting against \ac{TEE} objective;
\end{description}

Coherently with the framework described in the previous chapter, 
naturality indexes were computed for the three largest basins of each point of the three
clusters.
Probability distributions were built on their values, and the most
important results about them are commented in the following paragraph.
All less relevant results are included in \myAppendixNoSpace{appChap:allResults}.

\subsubsection{3D networks features}
As just mentioned, probability distributions of naturality indexes 
were created for the analyzed clusters, as the example shown in 
\myFig{fig:RaClusters_MOGLE} for Horton's $R_a$ index.

\begin{figure}
\myfloatalign
\includegraphics[width=0.75\columnwidth]{Images/MOGLE_HortonAreaRatio_clusters.pdf}
\caption[Statistical distribution of values of Horton's $R_a$ for the 
clusters of experiment \myExpOne]{Statistical distribution of
values of Horton's $R_a$ for the clusters of experiment \myExpOne.
The chosen clusters are the same of
\myFigNoSpace{fig:clusters_MOGLE}. On the $y$ there is the ratio
value, on $x$ axis the percentage value.}
\label{fig:RaClusters_MOGLE}
\end{figure}
 
As it is possible to see from \myFigNoSpace{fig:RaClusters_MOGLE},
the gray area represents the area of probability within the natural range.
Statistics about this area, for all Horton's ratios are summarized in 
\myTabNoSpace{tab:naturalityIndexes_mogle}.

\begin{table}[h]
\myfloatalign
\footnotesize
\begin{tabularx}{\textwidth}{p{0.38\textwidth}ccc}
\toprule
& \tableheadline{c}{Min TEE} & \tableheadline{c}{Compromise} &
\tableheadline{c}{Min EE} \\
\midrule
\tablefirstcol{p{0.38\textwidth}}{Number of cluster points} & $58$
& $382$ & $185$\\
\midrule
\tablefirstcol{p{0.38\textwidth}}{Area within natural range of
$R_b$ [\%]} & $32.73$ & $26.56$ & $29.59$  \\
 \midrule
 \tablefirstcol{p{0.38\textwidth}}{Area within natural range of
 $R_l$ [\%]} & $61.79$ & $60.41$ & $52.62$  \\
\midrule
\tablefirstcol{p{0.38\textwidth}}{Area within natural range of
$R_a$ [\%]} & $82.84$ & $61.31$ & $95.23$  \\
\midrule
\tablefirstcol{p{0.38\textwidth}}{Area within natural range of
$R_s$ [\%]} & $0$ & $0$ & $0$  \\
\bottomrule
\end{tabularx}
\caption[Naturality indexes statistics for
\myExpOneNoSpace]{Naturality indexes statistics for
\myExpOneNoSpace. The three cluster for which data are shown, are
the same of \myFigNoSpace{fig:clusters_MOGLE}.}
\label{tab:naturalityIndexes_mogle}
\end{table}

From the data in the table, no particular trends among the different clusters 
can be found, but a clear result appears: it seems that on the 
indexes assessing the 2D naturality of river networks, \ie $R_b$, 
$R_a$ and $R_l$, selected clusters do not perform so bad, being their 
probability higher than $50\%$ for $R_a$ and $R_l$, with also values 
higher than $90\%$ of probability, and maintaining values around $30\%$ 
for $R_b$.
On the contrary, it is evident that slope is really badly reproduced, since 
none of the values are inside the natural range.

The obtained result for this preliminary experiment is therefore partial, 
in the sense that it fails at reproducing the natural features of river 
networks with respect to the dimension of elevation and its consequent slopes.

A reason for that can be that the model, as it was set for this experiment had 
strong limitations:
\begin{itemize}
  \item the range variable controls can assume values in maybe is too 
  narrow and does not allows control variables to assume values which 
  could better satisfy the chosen optimal criteria and reproduce better 
  synthetic landscapes;
  \item the number of control variables maybe is too high: there is a 
  control variable for each \ac{DEM} cell. 
  Because of that, the relation between controls and objectives could be weak;
  \item the first two points might also affect the capability of the model to 
  reproduce the conflict among objectives and distinguish among the networks 
  related to their trade-offs; 
  \item the choice of the hydrological parameter of rainfall, equal to $0.1$ mm/h,
  may limit the system. In fact, in the nature, rivers shape the landscape 
  when they are in a condition of bank-full discharge.
  It can be supposed that a strong rainy event increase the system action 
  in transferring mass and shaping itself. Therefore, it might be possible that 
  $0.1$ mm/h is too low.  
\end{itemize}

\subsection{General conclusion on \myExpOne}
In light of the results obtained for this first experiment, it
appears that a large part of the optimized networks, obtained from
the Pareto front points, are characterized by acceptable values for
those indexes which evaluate the 2D features of a river network,
but show bad distributions with respect to Horton's slope
ratio.
Therefore, as in \citeauthor{paik:2011}'s \ac{GLE} model, there
are problems in properly representing the third dimension of river
networks.
As a consequence, the following question rises: 
What are the limitations causing the inability of simulating good
longitudinal profiles for river channels? 

The experiments which are analyzed in the following two sections
were made in order to answer this question.

%%%%%%%%%%%%%%%
\section{Increasing rainfall: \myExpTwo}
\label{sec:expRainy}

Experiment \myExpTwo is commented in this section. 
As written in the experiments description at the beginning of this
chapter, this second experiment differs from the previous one 
as for the value given to the rainfall input parameter.
It was increased from $0.1$ to $10$ mm/h, in order to consider 
a rainfall value typical of an average natural rainy event.
\footnote{Such a value can be considered as representative of 
extreme rainy events, as confirmed, for example, 
by the data gathered in \cite{haberlandt:2007}.}

Another important feature of this experiment is the discretization 
of control variables ranges. In fact, as underlined in the comments of the 
previous experiment, the chosen range limited the actions of control variables, 
therefore, a new setting was adopted:
\begin{itemize}
  \item in \myExpOne, the elevation of each cell could vary in the 
  range $[-10;+10]$ meters, discretized with the resolution of $1$ meter;
  \item in this \myExpTwo, the elevation of each cell is allowed to change 
  in the range $[-20;+20]$ meters, discretized with the resolution of 
  $2$ centimeters.
\end{itemize} 
This change could increase the complexity of the search for feasible 
control variables sets, but allows the control variables to have a bigger 
degree of freedom in their acceptable range, since they can assume a greater 
number of values within it, which can recombine in many different control sets.

All other features and model settings are equal to the ones of the 
first experiment: they were summarized in \myTab{tab:experimentsCommonAspects}, 
at the beginning of this chapter.
 
\subsection{Pareto front}
The Pareto front obtained as a result of the optimization of this
experiment is the one represented in
\myFigNoSpace{fig:PF_rainy10}.
\begin{figure}
\myfloatalign
\includegraphics[width=0.9\columnwidth]{Images/PF_rainy10.pdf}  
\caption[Pareto front obtained from \myExpTwo
experiment]{Pareto front obtained from \myExpTwo experiment.
The front was represented using AeroVis\copyright{}.}
\label{fig:PF_rainy10}
\end{figure}

\subsubsection{Algorithms performance}
As for the first experiment, some important features of the 
Pareto front shown in the previous
section are summarized in \myTabNoSpace{tab:paretoFrontRAINYdata}.
We remind that, as for feasibility, it represents the number of control sets
respecting the mass constraint over the total number of generated
sets, for each algorithm in the first three rows and for the total
number of function evaluations in the last row.
\begin{table}[h]
\myfloatalign
\begin{tabular}{lSS} 
\toprule {ALGORITHM} & {NUMBER OF FRONT POINTS} & {FEASIBILITY
($\%$)} \\
\midrule
{\ac{eNSGAII}} & 2145 & 62.05\\
\midrule
{\ac{GDE3}} & 115 & 44.61\\
\bottomrule
{\textbf{Total}} & 2260 & 53.33\\ 
\bottomrule
\end{tabular}
\caption[\myExpTwoNoSpace: Pareto front
features]{\myExpTwoNoSpace: Pareto front features.}
\label{tab:paretoFrontRAINYdata}
\end{table}

The largest contribution is therefore coming from \ac{eNSGAII} algorithm,
covering over the $94\%$ of the front.
\ac{eNSGAII} is also the algorithm having the largest feasibility
suggesting that is the one which best recognise the mass
constraint.

\subsubsection{Comparison with Random search}
Also for this experiment, the performance of \acp{EA} is compared to the one of 
Random search algorithm.
In this case, the performance of \ac{EA} is extremely superior, if compared to 
Random search. In fact, random search feasibility of finding control sets 
respecting the mass constraint is $7.2\times10^{-5}$ (on 50 millions 
of \ac{NFE}, only $36$ feasible control sets were found and all 
of them are dominated by the front given by \acp{EA}).
This fact confirms that \acp{EA} can better recognise the mass constraint and now, 
since the range for control variables is larger and the resolution is finer, 
random search is very limited in finding proper combinations of controls, while 
\acp{EA} lowered a bit their feasibility with respect to the one they had in 
\myExpOneNoSpace, especially \ac{GDE3}, but they work much better that random search.

\subsection{Analysis of the conflict} 
The conflict can be analyzed again looking at the front.
In particular:
\begin{itemize}
  \item it is very clear that \ac{EE} and \ac{EEE} are not in conflict at all;
  \item the conflict between \ac{TEE} and \ac{EE},supposed thanks to \cite{rodriguez:1992},
  is now more clear than in the first 
  experiment and the front is concave with respect to their projection;
  \item the relation between \ac{TEE} and \ac{EEL} is still unclear, but, despite 
  the points are still widespread, it seems there is not much conflict between them.
\end{itemize}
Given these considerations, it is possible to say that the new configuration 
of the model better allows to show the conflict among objectives, with respect to 
the first experiment.
Clustering and naturality index analysis might help the interpretation of this 
conflicting situation.

\subsection{Clustering and naturality indexes analysis}
In this experiment, five clusters were obtained with $k$-means
clustering and are represented with different colors in
\myFigNoSpace{fig:clusters_rainy10}.

\begin{figure}
\myfloatalign
\includegraphics[width=0.75\columnwidth]{Images/clusters_rainy10.pdf}
\caption[Pareto front clusters for \myExpTwo experiment]{Pareto
front clusters for \myExpTwo experiment. The upper colorbar
shows the colors of the different $5$ clusters.
Cluster \emph{Min\ac{TEE}} is the red one, \emph{Compromise} is the 
light blue one and \emph{Min\ac{EE}} the green one.
They are selected for naturality indexes analysis.
The front was represented using AeroVis \copyright{}.}
\label{fig:clusters_rainy10}
\end{figure}

As in the previous experiment, only three clusters were selected
for carrying out the assessment through naturality indexes.
They were again chosen in order to have two extreme
values and one compromise among \ac{TEE} and \ac{EE} objectives.
They are the following:
\begin{description}
  \item[\NoCaseChange{MinTEE}:] this cluster minimizes \ac{TEE}
  objective, counting against \ac{EE} objective;
  \item[\NoCaseChange{Compromise}:] this cluster represents the
  compromise between \ac{TEE} and \ac{EE} objectives;
  \item[\NoCaseChange{MinEE}:] this cluster minimizes \ac{EE}
  objective, counting against \ac{TEE} objective;
\end{description}

Furthermore, coherently with the previous experiment, 
distributions of naturality indexes were built, and the most
important results about them are commented in the following paragraph.
We remind that all less relevant results are included in 
\myAppendixNoSpace{appChap:allResults}.

\subsubsection{3D networks features}
Again, considering the values of naturality indexes and their 
probabilities of being in a natural range, the results is 
quantitatively different from the previous experiment, but not 
qualitatively.
Important values for Horton's ratio and Hack's law, for each cluster, 
are summarized in \myTabNoSpace{tab:naturalityIndexes_rainy}.

\begin{table}[h]
\myfloatalign
\footnotesize
\begin{tabularx}{\textwidth}{p{0.3\textwidth}ccc}
\toprule
& \tableheadline{c}{Min TEE} & \tableheadline{c}{Compromise} &
\tableheadline{c}{Min EE} \\
\midrule
\tablefirstcol{p{0.3\textwidth}}{Number of cluster points} & $55$ & $598$ &
$243$\\
\midrule
\tablefirstcol{p{0.3\textwidth}}{Area within natural range of $R_b$ [\%]}
 & $77.69$ & $54.87$ & $91.58$  \\
 \midrule
 \tablefirstcol{p{0.3\textwidth}}{Area within natural range of $R_l$ [\%]}
& $94.40$ & $81.83$ & $92.00$  \\
\midrule
\tablefirstcol{p{0.3\textwidth}}{Area within natural range of $R_a$ [\%]}
& $96.36$ & $96.24$ & $95.03$  \\
\midrule
\tablefirstcol{p{0.3\textwidth}}{Area within natural range of $R_s$ [\%]}
& $1.79$ & $0.3$ & $0$ \\
\midrule
\tablefirstcol{p{0.3\textwidth}}{Hack's law exponent}
& $0.67$ & $0.68$ & $0.66$  \\\bottomrule
\end{tabularx}
\caption[Naturality indexes statistics for \myExpTwo]{Naturality
indexes statistics for \myExpTwoNoSpace.}
\label{tab:naturalityIndexes_rainy}
\end{table}

Given the values in the table, it is possible to assert that, with the settings 
adopted for this experiment, again no particular trends and trade-offs 
among the clusters are shown, but the 2D features of river networks are really well 
represented, in the sense that their values of naturality indexes are for the 
majority inside natural ranges with high probability. 
The areas of Horton's ratios distributions ($R_b$, but mainly $R_a$ and $R_l$) 
inside natural ranges are very high and many o them higher than $90\%$.
Moreover, another index which describe the planar orgnization of river networks, 
\ie Hack's law exponent is very close to $0.6$, its
\enquote{natural} value. Its distributions are show in
\myFigNoSpace{fig:RAINY_HackExponents_clusters}.

\begin{figure}
\myfloatalign
\includegraphics[width=0.75\columnwidth]{Images/RAINY_HackExponents_clusters.pdf}
\caption[Statistical distribution of values of Hack's law exponent for the 
clusters of experiment \myExpTwo]{Statistical distribution of
values of Hack's law exponent for the clusters of experiment
\myExpTwo. The chosen clusters are the same of
\myFigNoSpace{fig:clusters_rainy10}. On the $y$ there is the ratio
value, on $x$ axis the percentage value.}
\label{fig:RAINY_HackExponents_clusters}
\end{figure}

On the other hand, the model is again limited when the slope, and therefore 
the third dimension should be reproduced. 
Comments on that are present in the last paragraph of this section, introducing 
the third experiment. 

\subsection{General conclusion on \myExpTwo}
It appears that the changes introduced in the model with respect to \myExpOne,
were useful in order to improve the performance on reproducing 
2D networks, but still 3D features are missing.
The hypothesis for that are, now:
\begin{itemize}
  \item the range of control variables, even if it was enlarged and a finer 
  resolution was adopted, remains a limitation for them and does not let 
  the landscape evolve completely freely;
  \item apart from the range of control variables, their number is the main issue.
  When the number of controls is too high, it looses the link with the objectives;
  \item if each \ac{DEM} cell is related to a single control value, different, even neighbouring, 
  cells of the \ac{DEM} evolve too independently from the closest ones. 
  Therefore, the surface of the landscape is not enough smoothed and this fact, 
  together with the action of depression filling algorithm, contributes in creating 
  sudden steps, peaks and unnatural slopes.
\end{itemize}
Mainly for the last reason, a spatial interpolator is integrated with the next, 
last experiment.

%%%%%%%%%%%%%
\section{Spatial interpolation: \myExpThree}
\label{sec:expIDW16}
Experiment \myExpThree is commented in this section. 
As written in the experiments description at the beginning of this
chapter, this third experiment includes the \ac{IDW} spatial
interpolator algorithm, explained in
\mySubsec{subs:spatialInterpolator}.\footnote{It is useful to
remind that the spatial interpolator was implemented in order to
let the optimizer create the concave bed profile and to reduce
the number of points the optimizer has to evaluate function
$f(\cdot)$ for, so that the connection between objectives and
controls is reinforced.}

\subsection{Spatial interpolator settings}
The following parameters were set for \ac{IDW}:
\begin{itemize}
  \item \myEmph{frequency}: $10$ cells.
  		It means that on a $51 \times 51$ cells \ac{DEM}, since the
  		border is excluded, the number of $f(\cdot)$ values to
  		optimize is reduced to $16$.
  		All the other elevations are obtained through interpolation;
  \item \myEmph{window}: $15$ cells.
  		It means that the elevation of each cell is interpolated only
  		on the optimized $f(\cdot)$ elevation values present in the
  		square mask composed by its neighbouring $225$ cells.
\end{itemize}

\subsection{Pareto front}
The Pareto front obtained as a result of the optimization of this
experiment is the one represented in \myFig{fig:paretoFrontIDW}.

\begin{figure}
\myfloatalign
\includegraphics[width=\columnwidth]{Images/AerovisViewOfPF_IDW.pdf}  
\caption[Pareto front obtained from \myExpThree
experiment]{Pareto front obtained from \myExpThree experiment.
The front was represented using AeroVis\copyright{}.}
\label{fig:paretoFrontIDW}
\end{figure}

\subsubsection{Algorithms performance}
Some important features of the Pareto front shown in the previous
section are summarized in \myTabNoSpace{tab:paretoFrontIDWdata}.
In particular, the number of points of the front each algorithm
contributed to is shown, together with the feasibility.
As for feasibility, it represents the number of control sets
respecting the mass constraint over the total number of generated
sets, for each algorithm in the first three rows and for the total
number of function evaluations in the last row.
\begin{table}[h]
\myfloatalign
\begin{tabular}{lSS} 
\toprule {ALGORITHM} & {NUMBER OF FRONT POINTS} & {FEASIBILITY
($\%$)} \\
\midrule
{\ac{eNSGAII}} & 396 & 4.72\\
\midrule
{\ac{GDE3}} & 654 & 18.83\\
\midrule
{\ac{OMOPSO}} & 55 & 0.31\\
\bottomrule
{\textbf{Total}} & 1105 & 8.19\\ 
\bottomrule
\end{tabular}
\caption[\myExpThreeNoSpace: Pareto front features]{\myExpThreeNoSpace:
Pareto front features.}
\label{tab:paretoFrontIDWdata}
\end{table}

The largest contribution is therefore from \ac{GDE3} algorithm,
covering over the $59\%$ of the front.
\ac{GDE3} is also the algorithm having the largest feasibility
suggesting that is the one which best recognise the mass
constraint.

As in the previous two experiments, also the results of this
experiment support and justify the use of \ac{GA}s.
In fact, the following comparison with random control generation
was performed:
\begin{itemize}
  \item a sample of $400\,000$ different random values for each
  control was generated;
  \item $13.6$ millions of different control sets were created as
  combinations of the previously generated controls by the use of
  Saltelli's method implemented in the \ac{MOEA} framework. The
  magnitude order of control sets is therefore comparable to the
  \ac{NFE} for the \ac{GA}s, which was set equal to $10$ millions,
  as shown in \myTabNoSpace{tab:experimentsEAsettings}.
  \item Among all the control sets generated, none of them was
  able to respect the mass constraint.
\end{itemize}
This fact, again, confirms that evolutionary algorithms are
effective in understanding the mass constraint and produce proper
control sets.

\subsubsection{A comparison with \myExpTwo}
It is possible to compare the Pareto fronts obtained for 
\myExpTwo and \myExpThreeNoSpace.
They are both represented in \myFigNoSpace{fig:IDW_vs_rainy10}.

\begin{figure}
\myfloatalign
\includegraphics[width=0.75\columnwidth]{Images/IDW_vs_rainy10.pdf}  
\caption[Comparison between \myExpTwo and \myExpThree Pareto fronts]
{Comparison between the Pareto fronts of \myExpTwo without the
interpolator and \myExpThree with \ac{IDW} interpolatro.
The front was represented using AeroVis\copyright{}.}
\label{fig:IDW_vs_rainy10}
\end{figure}

In particular, it is possible to say that as for the objectives values, 
the pareto Front obtained in \myExpTwo is dominant. 
Nevertheless the following thing is important:
the front obtained with the interpolator is much more extended.
This fact confirms the comments written at the end of the previous 
section, \ie that a range for control variables is a limitation 
in reproducing very different control sets and representing the 
conflict among the objectives. 
Now, since the interpolator is directly applied to the \ac{DEM} 
surface, and the surface itself is directly optimized, a bigger 
degree of freedom is given to the optimizer, allowing it to explore 
very different areas in the objectives space.


\subsubsection{Analyzing the conflict}
Looking at the Pareto front, it appears that the conflict among
objectives, already treated in the previous two sections, is now 
more present and clear.
This evidence supports the hypothesis of conflicting objectives,
since it now clearly emerges.
Moreover, the main conflicts are the ones expected:
\begin{itemize}
  \item \ac{TEE} and \ac{EE} are conflicting;
  \item \ac{TEE} is not conflicting with its variance \ac{EEL},
  but the front points on the quarter (\ac{TEE}, \ac{EEL}) are
  spread;
  \item \ac{EE} is not conflicting with its variance \ac{EEE};
  \item as a consequence of the previous observations, \ac{TEE} is
  conflicting also with \ac{EEE}, and \ac{EEE} with \ac{EEL}.
\end{itemize} 
According to the methodology adopted for analyzing the previous
two experiments, considering the conflict between \ac{TEE} and
\ac{EE} as the most evident, it is possible to exclude \ac{EEE}
from the representation of the front being it concordant with
\ac{EE}. We proceed with the clustering and the naturality indexes
analysis, as commented in the following subsections.


\subsection{Clustering and naturality indexes analysis}
\label{subs:IDWclustering}
In this experiment, fifteen clusters were obtained with $k$-means
clustering and are represented with different colors in
\myFigNoSpace{fig:paretoFrontIDWclusters}.

\begin{figure}
\myfloatalign
\includegraphics[width=0.8\columnwidth]{Images/AerovisView_IDWcluster.pdf}
\caption[Pareto front clusters for \myExpThree experiment]{Pareto
front clusters for \myExpThree experiment. The upper colorbar
shows the colors of the different $15$ clusters.
Clusters \emph{Min\ac{TEE}}, \emph{Compromise} and
\emph{Min\ac{EE}} are surrounded by a coloured circle, since they
are the selected clusters for naturality indexes analysis.
The front was represented using AeroVis \copyright{}.}
\label{fig:paretoFrontIDWclusters}
\end{figure}

As in the previous experiments, only three clusters were selected
for carrying out the analysis through naturality indexes and
again, the three clusters were chosen in order to have two extreme
clusters and one compromise.
They are the following:
\begin{description}
  \item[\NoCaseChange{MinTEE}:] this cluster minimizes \ac{TEE}
  objective, counting against \ac{EE} objective. This class was
  chosen in spite of class $9$, the one composed just by three
  green points above class MinTEE, in order to have a larger
  amount of points and therefore basins for the
  naturality indexes analysis;
  \item[\NoCaseChange{Compromise}:] this cluster represents the
  compromise between \ac{TEE} and \ac{EE} objectives;
  \item[\NoCaseChange{MinEE}:] this cluster minimizes \ac{EE}
  objective, counting against \ac{TEE} objective;
\end{description}

Coherently with the previous experiments, naturality indexes were
computed for the three largest basins of each point of the three
clusters.
Probability distributions were built on their values, and the most
important results about them are commented in the following two
paragraphs.
All less relevant results are included in \myAppendixNoSpace{appChap:allResults}.

\subsubsection{2D networks features}
Some first comments about the landscape and river networks
generated with this third experiment may be done considering the
indicators which refer to the 2D features of river networks, \ie
all apart from $R_s$ (which includes the third dimension in the
evaluation of the slope).

In general, no significant improvements are observed when
considering the distributions of Horton's ratios $R_b$, $R_l$ and
$R_a$.
Also, no significant trends can be identified for those indexes
while moving from one extreme cluster to another passing through
the compromise.
Anyway, the following elements appear:
\begin{itemize}
  
  \item usually the area of $R_b$, $R_l$ and $R_a$ which is inside
  the natural range is larger for clusters \emph{Compromise} and
  \emph{MinEE}. Anyway, the index distributions for those clusters
  have a bigger standard deviation, if compared to the ones for
  \emph{MinTEE} cluster, therefore the distribution is not
  concentrated and some basins are far from the natural range. As
  a consequence, it is not possible to affirm which cluster
  performs better with respect to those three Horton's ratios;
  
  \item on the contrary, considering Hack's exponent, as shown in
  \myFigNoSpace{fig:HackClusters_IDW}, the average value is closer
  to the natural value $0.6$ for \emph{MinEE} cluster, which also
  shows a smaller standard deviation, if compared to the others.
  It seems so that \emph{MinEE} performs better than
  \emph{Compromise} and \emph{MinTEE}, which is the one whith the
  worst behaviour;
  
  \item in general, river networks appear worst and less realistic
  than the ones obtained for the previous experiment. An example
  of them is provided in \myFig{fig:networkExample}. This fact is
  for sure due to the combined effect of \ac{IDW} and depression
  filling algorithms, which create almost flat areas in some
  points of the \ac{DEM} and, in this way, act against more
  articulated and branched river networks.
\end{itemize}

\begin{figure}
\myfloatalign
\includegraphics[width=\columnwidth]{Images/networkExampleTEE.pdf}  
\caption[Example of river network for \myExpThree]{Example of
river networks for \myExpThree. Black lines represent river
networks exctracted from the \ac{DEM} of the centroid point of
\emph{MinTEE} cluster.}
\label{fig:networkExample}
\end{figure}

\begin{figure}
\myfloatalign
\includegraphics[width=0.75\columnwidth]{Images/HackExponents_clusters.pdf}
\bigskip

\footnotesize
\begin{tabularx}{\textwidth}{p{0.25\textwidth}ccc}
\toprule
& \tableheadline{c}{Min TEE} & \tableheadline{c}{Compromise} &
\tableheadline{c}{Min EE} \\
\midrule
\tablefirstcol{p{0.25\textwidth}}{Number of cluster points} & $40$
& $118$ & $40$\\
\midrule
\tablefirstcol{p{0.25\textwidth}}{Sample mean and standard
deviation} & $0.7040 \pm 0.06832$ & $0.5287 \pm 0.07906$ & $0.6465
\pm 0.05714$ \\
\bottomrule
\end{tabularx}

\caption[Statistical distribution of values of Hack's law
exponent for the clusters of experiment
\myExpThreeNoSpace]{Statistical distribution of values of
Hack's law exponent for the clusters of experiment
\myExpThreeNoSpace. The chosen clusters are the same of
\myFigNoSpace{fig:paretoFrontIDWclusters}. On the $y$
there is the ratio value, on $x$ axis the percentage value. The
table below integrates the graphics with simple statistics.}
\label{fig:HackClusters_IDW}
\end{figure}

As a consequence of the commented observation, we conclude that
river networks simulated within \myExpThreeNoSpace do not show
particular pattern and trends while moving along the Pareto front
points when considered as 2D networks without the elevation
dimension.

\subsubsection{Slope and river longitudinal profiles}
The interesting results of this experiment come when analyzing
the 3D features of river networks.
That means looking at the distributions of $R_s$ index for the
three clusters; they are represented in
\myFigNoSpace{fig:HortonSlopeClusters_IDW}.
Even if the part of the distribution inside the natural
range does not change very much among the clusters, and it is
always below $60\%$, big differences exist among the clusters:
\begin{itemize}  
  \item $R_s$ distribution for \emph{MinTEE} cluster (left side
  plot in \myFigNoSpace{fig:HortonSlopeClusters_IDW}) has over
  $50\%$ of its area included in the natural range.
  Anyway, the fact that its average value is approximately $13$
  and its standard deviation is the largest, suggests that the
  distribution is very widespread and values with high probability
  are very far from the natural range $[1.5;\ 3.5]$ and they lie
  on the portion of distribution which is outside the plotted one;
  
  \item $R_s$ distribution for \emph{Compromise} (central plot in
  \myFigNoSpace{fig:HortonSlopeClusters_IDW}) shows that less than
  $50\%$ of its area is included in the natural range (about $44\%$).
  Anyway, this distribution is better than the one for
  \emph{MinTEE} cluster, because the average value of $2.27$ is
  closer to the mean value of the natural range and its standard
  deviation is much smaller, if compared to the one for
  \emph{MinTEE} cluster.
  This means that almost all the values which are not inside the
  natural range are close to it;
  
  \item $R_s$ distribution for \emph{MinEE} (right side plot in
  \myFigNoSpace{fig:HortonSlopeClusters_IDW}) is the one which
  better approximates the natural range.
  In fact, it has the largest area inside it, its average value is
  close to the center of the natural range and its variance is the
  smallest among the distributions of the three clusters.
\end{itemize}

\begin{figure}
\myfloatalign
\includegraphics[width=0.75\columnwidth]{Images/HortonSlopeRatio_clusters.pdf}
\bigskip

\footnotesize
\begin{tabularx}{\textwidth}{p{0.3\textwidth}ccc}
\toprule
& \tableheadline{c}{Min TEE} & \tableheadline{c}{Compromise} &
\tableheadline{c}{Min EE} \\
\midrule
\tablefirstcol{p{0.3\textwidth}}{Number of cluster points} & $40$
& $118$ & $40$\\
\midrule
\tablefirstcol{p{0.3\textwidth}}{Sample mean and standard
deviation} & $13.64 \pm 13.25$ & $2.270 \pm 3.523$ & $2.117
\pm 2.241$ \\
\midrule
\tablefirstcol{p{0.3\textwidth}}{Probability within natural
range} & $0.5184$ & $0.4387$ & $0.5699$\\
\bottomrule
\end{tabularx}

\medskip
\flushleft{ \footnotesize 
\textbf{Note on the left plot}: the sharp distribution showed in
the first plot is due to the distribution interpolator algorithm
used. In fact, its Matlab\copyright{} implementation builds the
statistical distribution of the dataset, then samples it with a
fixed amount of point. Since the range of values covered by the
Horton's slope ratio in this case is very wide, the sampling step
is wide too. In this particular case is about $0.5$: this explain
the shape of the plot.}
\medskip

\caption[Statistical distribution of Horton's slope
ratio for the clusters of experiment
\myExpThreeNoSpace]{Statistical distribution of Horton's slope
ratio for the clusters of experiment
\myExpThreeNoSpace. The chosen clusters are the same of
\myFigNoSpace{fig:paretoFrontIDWclusters}.}
\label{fig:HortonSlopeClusters_IDW}
\end{figure}

As a consequence, it is possible to state that the cluster which
minimizes the objective containing the 3D term of slope, \ie
\ac{EE}, is also the one which best performs according to Horton's
slope ratio, while the performance gets worse when moving along
the Pareto front toward the opposite extreme cluster, \ie the ones
that minimizes \ac{TEE}.

In order to verify if the performances on $R_s$ index really
reflect on networks patterns, rivers longitudinal profiles were
analyzed.
Specifically, one significant Pareto front point for each cluster
was taken as sample, \ie the one closest to the cluster centroid.
The main channel of the largest network developing on the \ac{DEM}
represented by those front points was identified and its
longitudinal profile is represented in
\myFigNoSpace{fig:profilesIDW}.

\begin{figure}
\myfloatalign
\includegraphics[width=0.5\columnwidth]{Images/profilesIDW.pdf}  
\caption[Rivers longitudinal profiles for \myExpThree
experiment]{Rivers longitudinal profiles for \myExpThree
experiment. 
Starting from the top, the three plots represent the longitudinal 
profile of the main channel for the largest network in, 
respectively, \emph{Min\ac{TEE}}, \emph{Compromise} and 
\emph{Min\ac{EE}} clusters centroids.
The straight line in each sub-plot connects the spring to the outlet.
Pay attention to the different scales on $x$- and
$y$-axes.}
\label{fig:profilesIDW}
\end{figure}

As it is possible to appreciate from the figure, the only profile
showing some promising results with respect to concavity,
 is the one belonging to \emph{MinEE}
cluster, which is also the one with the best results with respect
to $R_s$.
On the other hand, the one belonging to \emph{MinTEE} can be
considered the most convex.
Since natural profiles are characterized by concavity, in our
experiment \emph{MinEE} cluster is the one which better
approximates natural patterns with respect to that feature.
Moreover, even if only one significant profile for each cluster
was represented here, the analysis of other points confirmed the
just affirmed comments.

Even if the result is good, further improvements can be done. 
In fact, as it is possible to see again from
\myFig{fig:profilesIDW}, almost flat areas and sudden steps are
present.
They are probably due to depression filling algorithm and the
settings chosen for \ac{IDW} interpolator algorithm, or to the
algorithm itself.

\subsection{General conclusion on \myExpThree}
The summarizing comment on this last experiment is positive.
Despite a not good performance of the model with respect
to the 2D features of river networks independently from the
objectives values, very good results were obtained with respect to
the third dimension represented by rivers longitudinal profiles.
It seems that the integration of a spatial interpolator in the
model reduces its limitations and helps the optimizer to interpret
the objectives, especially regarding the ones including the slope
term \ie \ac{EE} and \ac{EEE}.
With respect to 3D river networks, this is the first promising attempt
 in simulating concave longitudinal profiles of river,
if compared to the previous studies mentioned in
\mySubsec{subs:optimalityApproach} studies.
